在众多计算机视觉应用中，评估非刚性形状的相似性是一项基本任务。在这里，我们提出了一种新型的公理方法，以匹配跨形状的相似区域。匹配相似区域被配制为与Laplace-Beltrami操作员（LBO）密切相关的操作员的对齐。所提出方法的主要新颖性是考虑具有多个指标的多种歧管上定义的差分运算符。指标的选择与基本形状属性有关，同时考虑不同指标下的同一歧管，可以将其视为从不同角度分析了基本歧管。具体而言，我们检查了标准不变的度量和相应的尺度不变的拉普拉斯 - 贝特拉米操作员（Si-LBO）以及常规度量和常规LBO。我们证明，规模不变的度量强调了铰接形状中重要语义特征的位置。因此，Si-LBO的截断光谱更好地捕获了局部弯曲的区域，并补充了常规LBO截断光谱中封装的全局信息。我们表明，在标准基准测试时，将这些双光谱匹配的公理框架优于竞争的公理框架。我们介绍了一个新的数据集，并将所提出的方法与跨数据库配置中的基于最先进的学习方法进行了比较。具体而言，我们表明，在对一个数据集进行培训并在另一个数据集上进行测试时，提出的不涉及培训的公理方法优于深度学习替代方案。

虚拟网格是在线通信的未来。服装是一个人身份和自我表达的重要组成部分。然而，目前，在培训逼真的布置动画的远程介绍模型的必需分子和准确性中，目前无法使用注册衣服的地面真相数据。在这里，我们提出了一条端到端的管道，用于建造可驱动的服装代表。我们方法的核心是一种多视图图案的布跟踪算法，能够以高精度捕获变形。我们进一步依靠跟踪方法生产的高质量数据来构建服装头像：一件衣服的表达和完全驱动的几何模型。可以使用一组稀疏的视图来对所得模型进行动画，并产生高度逼真的重建，这些重建忠于驾驶信号。我们证明了管道对现实的虚拟电视应用程序的功效，在该应用程序中，从两种视图中重建了衣服，并且用户可以根据自己的意愿进行选择和交换服装设计。此外，当仅通过身体姿势驱动时，我们表现出一个具有挑战性的场景，我们可驾驶的服装Avatar能够生产出比最先进的面包质量明显更高的逼真的布几何形状。

深度估计是需要对环境的3D评估的广大应用程序的基石，例如机器人，增强现实和自主驱动来命名几个。深度估计的一个突出技术是立体声匹配，其具有多种优点：它被认为比其他深度传感技术更容易进入，可以实时产生密集的深度估计，并从近年来深度学习的进步中受益匪浅。然而，用于立体图像的深度估计的当前技术仍然遭受内置缺点。为了重建深度，立体声匹配算法首先在应用几何三角测量之前估计左图像和右图像之间的视差图。一个简单的分析表明，深度误差与对象距离相当成比例。因此，恒定的差异误差被转换为远离相机的物体的大深度误差。为了缓解这种二次关系，我们提出了一种简单但有效的方法，使用细化网络进行深度估计。我们展示了分析和经验结果表明所提出的学习程序减少了这种二次关系。我们评估了众所周知的基准和数据集的提出的细化程序，如演唱者和基提数据集，并在深度精度度量中展示了显着的改进。

Neural Representations have recently been shown to effectively reconstruct a wide range of signals from 3D meshes and shapes to images and videos. We show that, when adapted correctly, neural representations can be used to directly represent the weights of a pre-trained convolutional neural network, resulting in a Neural Representation for Neural Networks (NeRN). Inspired by coordinate inputs of previous neural representation methods, we assign a coordinate to each convolutional kernel in our network based on its position in the architecture, and optimize a predictor network to map coordinates to their corresponding weights. Similarly to the spatial smoothness of visual scenes, we show that incorporating a smoothness constraint over the original network's weights aids NeRN towards a better reconstruction. In addition, since slight perturbations in pre-trained model weights can result in a considerable accuracy loss, we employ techniques from the field of knowledge distillation to stabilize the learning process. We demonstrate the effectiveness of NeRN in reconstructing widely used architectures on CIFAR-10, CIFAR-100, and ImageNet. Finally, we present two applications using NeRN, demonstrating the capabilities of the learned representations.

A core process in human cognition is analogical mapping: the ability to identify a similar relational structure between different situations. We introduce a novel task, Visual Analogies of Situation Recognition, adapting the classical word-analogy task into the visual domain. Given a triplet of images, the task is to select an image candidate B' that completes the analogy (A to A' is like B to what?). Unlike previous work on visual analogy that focused on simple image transformations, we tackle complex analogies requiring understanding of scenes. We leverage situation recognition annotations and the CLIP model to generate a large set of 500k candidate analogies. Crowdsourced annotations for a sample of the data indicate that humans agree with the dataset label ~80% of the time (chance level 25%). Furthermore, we use human annotations to create a gold-standard dataset of 3,820 validated analogies. Our experiments demonstrate that state-of-the-art models do well when distractors are chosen randomly (~86%), but struggle with carefully chosen distractors (~53%, compared to 90% human accuracy). We hope our dataset will encourage the development of new analogy-making models. Website: https://vasr-dataset.github.io/

Micron-scale robots (ubots) have recently shown great promise for emerging medical applications, and accurate control of ubots is a critical next step to deploying them in real systems. In this work, we develop the idea of a nonlinear mismatch controller to compensate for the mismatch between the disturbed unicycle model of a rolling ubot and trajectory data collected during an experiment. We exploit the differential flatness property of the rolling ubot model to generate a mapping from the desired state trajectory to nominal control actions. Due to model mismatch and parameter estimation error, the nominal control actions will not exactly reproduce the desired state trajectory. We employ a Gaussian Process (GP) to learn the model mismatch as a function of the desired control actions, and correct the nominal control actions using a least-squares optimization. We demonstrate the performance of our online learning algorithm in simulation, where we show that the model mismatch makes some desired states unreachable. Finally, we validate our approach in an experiment and show that the error metrics are reduced by up to 40%.

A master face is a face image that passes face-based identity authentication for a high percentage of the population. These faces can be used to impersonate, with a high probability of success, any user, without having access to any user information. We optimize these faces for 2D and 3D face verification models, by using an evolutionary algorithm in the latent embedding space of the StyleGAN face generator. For 2D face verification, multiple evolutionary strategies are compared, and we propose a novel approach that employs a neural network to direct the search toward promising samples, without adding fitness evaluations. The results we present demonstrate that it is possible to obtain a considerable coverage of the identities in the LFW or RFW datasets with less than 10 master faces, for six leading deep face recognition systems. In 3D, we generate faces using the 2D StyleGAN2 generator and predict a 3D structure using a deep 3D face reconstruction network. When employing two different 3D face recognition systems, we are able to obtain a coverage of 40%-50%. Additionally, we present the generation of paired 2D RGB and 3D master faces, which simultaneously match 2D and 3D models with high impersonation rates.

The use of needles to access sites within organs is fundamental to many interventional medical procedures both for diagnosis and treatment. Safe and accurate navigation of a needle through living tissue to an intra-tissue target is currently often challenging or infeasible due to the presence of anatomical obstacles in the tissue, high levels of uncertainty, and natural tissue motion (e.g., due to breathing). Medical robots capable of automating needle-based procedures in vivo have the potential to overcome these challenges and enable an enhanced level of patient care and safety. In this paper, we show the first medical robot that autonomously navigates a needle inside living tissue around anatomical obstacles to an intra-tissue target. Our system leverages an aiming device and a laser-patterned highly flexible steerable needle, a type of needle capable of maneuvering along curvilinear trajectories to avoid obstacles. The autonomous robot accounts for anatomical obstacles and uncertainty in living tissue/needle interaction with replanning and control and accounts for respiratory motion by defining safe insertion time windows during the breathing cycle. We apply the system to lung biopsy, which is critical in the diagnosis of lung cancer, the leading cause of cancer-related death in the United States. We demonstrate successful performance of our system in multiple in vivo porcine studies and also demonstrate that our approach leveraging autonomous needle steering outperforms a standard manual clinical technique for lung nodule access.

The field of emergent communication aims to understand the characteristics of communication as it emerges from artificial agents solving tasks that require information exchange. Communication with discrete messages is considered a desired characteristic, for both scientific and applied reasons. However, training a multi-agent system with discrete communication is not straightforward, requiring either reinforcement learning algorithms or relaxing the discreteness requirement via a continuous approximation such as the Gumbel-softmax. Both these solutions result in poor performance compared to fully continuous communication. In this work, we propose an alternative approach to achieve discrete communication -- quantization of communicated messages. Using message quantization allows us to train the model end-to-end, achieving superior performance in multiple setups. Moreover, quantization is a natural framework that runs the gamut from continuous to discrete communication. Thus, it sets the ground for a broader view of multi-agent communication in the deep learning era.

在视频分析中，背景模型具有许多应用，例如背景/前景分离，变更检测，异常检测，跟踪等。但是，尽管在静态相机捕获的视频中学习这种模型是一项公认的任务，但在移动相机背景模型（MCBM）的情况下，由于算法和可伸缩性挑战，成功率更加重要。由于相机运动而产生。因此，现有的MCBM在其范围和受支持的摄像头类型的限制中受到限制。这些障碍还阻碍了基于深度学习（DL）的端到端解决方案的这项无监督的任务。此外，现有的MCBM通常会在典型的大型全景图像或以在线方式的域名上建模背景。不幸的是，前者造成了几个问题，包括可扩展性差，而后者则阻止了对摄像机重新审视场景先前看到部分的案例的识别和利用。本文提出了一种称为DEEPMCBM的新方法，该方法消除了上述所有问题并实现最新结果。具体而言，首先，我们确定与一般和DL设置的视频帧联合对齐相关的困难。接下来，我们提出了一种新的联合一致性策略，使我们可以使用具有正则化的空间变压器网，也不是任何形式的专业化（且不差异）的初始化。再加上在不破坏的稳健中央矩（从关节对齐中获得）的自动编码器，这产生了一个无端到端的无端正规化MCBM，该MCBM支持广泛的摄像机运动并优雅地缩放。我们在各种视频上展示了DEEPMCBM的实用程序，包括超出其他方法范围的视频。我们的代码可在https://github.com/bgu-cs-vil/deepmcbm上找到。